Guideline: In More Details - Define The Test Environment (MTP)
Relationships
Main Description

While each test level can organise the detailed description of the environment it requires, there are reasons to start this at the master test plan level:

  • Arranging, preparing and setting up a test environment often takes a lot of time, so it is best to start doing this at the earliest possible stage.
  • An environment is (very) costly; not every test level has a budget for an environment of its own. In this case, limiting the number of test environments by sharing the environment with another test level may be an option. This requires good alignment and planning.
  • The costs of the test environment must be included in the total estimated effort of the master test plan.
  • Another option sometimes used is to turn the acceptance environment into the production environment after the test. The disadvantage is that, once the system is taken into production, there is no more environment for acceptance testing changes or new releases. This, too, can be decided on the basis of the master test plan.
  • Often a test level is given one test environment by default. This does not take into account that a test level often tests a range of quality characteristics; the testing of each may have very different requirements for the environment. For instance, a performance test requires a production-like environment, but a user-friendliness test is perhaps better executed in a prototype-like environment, or in a usability lab with video cameras and transparent mirror walls. This requires early arrangement of the test environments.
  • The execution of test levels is also perceived as sequential: “after the system test of version 0.5.2, this version can be transferred to the acceptance environment”. Such sequential execution does not take adequate account of the need to execute tests as early as possible. For instance, the user-friendliness of the screens can be tested in the user acceptance test before the system test does a functional test of these screens. This can be done in the development environment or the acceptance environment, if the latter is available at this early stage. A conscious choice is then made between obtaining earlier test results and lower representativeness. Vice versa, the builders may assess performance as a high risk and wish to perform an (early) performance test. The production-like acceptance environment is most suitable to this end. This requires early and prior alignment.
  • All these test levels and their environments may quickly result in a considerable number of test environments. The question is: who manages these environments? This can be done by a system management or infrastructure department, or a Testing line organisation. To ensure good communication, the managing party must have a (fixed) point of contact within the test process. This may be the test manager, but it is better to have a (test) infrastructure expert at the overall level in the test process.